CityLearn: A Tutorial on Reinforcement Learning Control for Grid-Interactive Efficient Buildings and Communities

Kingsley Nweye

Description

The CityLearn tutorial at RLEM'23 will help participants get acquainted with the CityLearn OpenAI Gym environment, developed for easy implementation and benchmarking of control algorithms, e.g., rule-based control, model predictive control or deep reinforcement learning control in the demand response, building energy and grid-interactive community domain. By the end of the tutorial, participants will learn how to design their own simple or advanced control algorithms to provide energy flexibility, and acquire familiarity with the CityLearn environment for extended use in personal projects.

Learning Outcomes

The primary learning outcome for participants is to gain familiarity with CityLearn environment, its application programming interface (API) and dataset offerings for extended use in academic research or personal projects. Other secondary outcomes are to:

1. Understand how electrification, distributed energy resources e.g. batteries, photovoltaic (PV) systems and smart controls provide a promising pathway to decarbonization and energy flexibility.
2. Learn how to design and optimize their own rule-based control (RBC) agent for battery management using readily available knowledge of a building's energy use.
3. Identify the challenges surrounding the generalizability of an RBC agent and how reinforcement learning (RL) can mitigate these challenges.
4. Train their own RL Tabular Q-Learning algorithm.
5. Evaluate the performance of a standard model-free deep RL algorithm in optimizing key performance indicators (KPIs) that are targeted at quantifying energy flexibility, environmental and economic costs.
6. Learn the effect of different control algorithms and their parameters in improving these KPIs.
7. Make submission to The CityLearn Challenge 2023.

Target Audience

The target audience for this tutorial includes the following:

• Academic, private and commercial researchers or professionals that are interested in sustainable artificial intelligence (AI)-related pathways to electrification and building decarbonization.
• I enthusiasts with at least beginner level of expertise in programming or data science whom may have or pick interest in solving control problems or are interested in learning about a new reinforcement learning (RL) environment that deviates from popular simpler problems e.g. the Atari, MuJoCo and Box2D Gym environments, to real-world problems with urgency like climate change mitigation and decarbonization of the electric grid infrastructure and building end-uses.

Prerequisites

The CityLearn tutorial has a fairly low entry level and participants do not need to have prior experience in reinforcement learning (RL) nor use of a Gym environment. However, participants need to have at least, beginner knowledge in Python or other similar high-level scripting language. Also, participants should have a computer that is able to launch a Google Colab notebook in the browser or a Jupyter notebook locally.

Program

The workshop is divided into the following parts (note that durations are flexible):

References

• CityLearn Tutorial Notebook
• CityLearn GitHub
• CityLearn Documentation

 Overview  Program